Blind source separation of complex signals composed of reconnaissance, interference, detection, and communication signals is the first step in the processing of received signals in the integrated reconnaissance, interference, detection, and communication system, which requires higher accuracy of blind source separation. The traditional blind source separation method based on independent component analysis has such defects as the easy acquisition of local optimums and poor separation performance. To solve these problems, this paper proposes an improved pigeon-inspired optimization algorithm for blind source separation of complex signals. A location factor is added to the map and compass operators of the pigeon-inspired optimization algorithm, and a compression factor is added to the landmark operator. In this way, the global exploration ability in the early stage and the local search accuracy in the later stage of the algorithm are balanced, and the problems of the easy fall into local optimums and premature convergence are solved. The optimization ability and convergence speed of the algorithm are improved. Simulations show that the algorithm proposed in this paper can better separate complex signals under low and strong noise. Compared with the traditional independent component analysis method, the algorithm has better separation performance and convergence speed.